Three lemmas on the dynamic cavity method 



Erik Aurell 

Department of Computational Biology, AlbaNova University Centre, 106 91 Stockholm, Swedei^ 

Hamed Mahmoudi 

Department of Information and Computer Science, Aalto University, Finland 
(Dated: January 19, 2013) 

We study the dynamic cavity method for dilute kinetic Ising models with synchronous update 
rules. For the parallel update rule we find for fully asymmetric models that the dynamic cavity 
equations reduce to a Markovian dynamics of the (time-dependent) marginal probabilities. For the 
random sequential update rule, also an instantiation of a synchronous update rule, we find on the 
other hand that the dynamic cavity equations do not reduce to a Markovian dynamics, unless an 
additional assumption of time factorization is introduced. For symmetric models we show that a 
fixed point of ordinary Belief propagation is also a fixed point of the dynamic cavity equations in 
the time factorized approximation. For clarity, the conclusions of the paper are formulated as three 
lemmas. 

PACS numbers: 68.43.De, 75.10.Nr, 24.10.Ht 



I. INTRODUCTION 



For diverse applications to information theory, artificial intelligence and other fields, as well as in the physics of 
dilute spin glasses, much attention has been given over the last decade to a class of distributed computational schemes 
known as iterative decoding, Belief Propagation (BP) or the cavity method [3[TT]. These methods determine the 
marginals of Markovian random fields, where the dependency structure is encoded by a factor graph. The method is 
exact if the factor graph is a tree, and often very accurate if the factor graph is locally tree-like. The prime examples 
of such locally tree-like graphs graphs are on random graphs or random hyper-graphs, which underlie, for instance, 
LPDC codes, random graph coloring and and random fc-satisfiability, and the dilute Sherrington-Kirkpatrick spin 
glass. 

A general feature of most applications to date of these schemes is that they target marginals of Boltzmann-Gibbs 
measures, which describe physical systems in equilibrium. Such measures are also the stationary state of (families of) 
Monte Carlo (MC) schemes, where the update rules obey detailed balance. The main advantage of BP is then that 
it is (typically) many times faster than MC, and therefore the preferred choice when marginals of Boltzmann-Gibbs 
measures have to be computed both accurately and efficiently. 

A synchronous update Monte-Carlo scheme to simulate e.g. an Ising spin glass can be visualized as a tower of 
variable sets, where each horizontal layer represent the spins at some time t, and where the links between the layers 
encode the dependences in the update rules. Such a description is not limited to equilibrium physics, but extends to 
update rules which do not obey detailed balance. 

The question then naturally poses itself whether a distributed computational scheme can be found which computes 
the marginal distributions of such factor graphs. Kanoria and Montanari in [3] showed that this is the case for majority 
dynamics on trees, while Neri and Bolle in [7] showed that given an assumption which we will call time factorization 
the (asymmetric, non-equilibrium) Ising spin glass with parallel update rule also leads to a BP-like scheme. In pQ we 
extended the Neri-Bolle approach to a sequential update rule, and showed that it gives indeed in many cases very 
accurate predictions of the marginals of stationary non-equilibrium states. 

These results on dynamic cavity method are, we believe, quite important, as potentially pointing to a new class 
of general, accurate and efficient approximation schemes in non-equilibrium systems. They therefore deserve further 
study, outlining when and how they work, and when they don't. In this contribution we will address the following 
aspects of the systems studied in [7] and pQ: (i) does the dynamic cavity reduce to a Markovian dynamics if the 
underlying graph is fully asymmetric? (ii) is there a difference depending on which update rule (parallel or sequential) 
is used? (Hi) what is the relation between the dynamic cavity method and ordinary BP if the underlying graph is 
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symmetric, and hence describes an equilibrium system? 

The answers to the first two questions, which we formulate as Lemma 1 and Lemma 2 below, are that for fully 
asymmetric graphs and the parallel update rule, the dynamic cavity equations do reduce to a Markovian dynamics, 
without additional assumptions, but for the sequential update rule this is not so. We remark that for tractability 
the dynamic cavity equations must be reduced to a Markovian dynamics, as otherwise one would have to keep track 
of the whole history in a simulation. Therefore, Lemma 1 and Lemma 2 also mean that for parallel updates on a 
fully asymmetric graph, the reduced cavity equations are in a certain sense exact, and hold both for transients and 
stationary states, while for sequential updates this is not so. Indeed, in [Tj we only found good agreement between 
the reduced cavity equations and MC under sequential updates for the stationary states, but not for transients. The 
answer to the third question, which was already stated in [7], is that a fixed point of BP on an equilibrium Ising 
model can be extended to a fixed point of dynamic cavity method for the same model. As no proof of this result has 
appeared in the literature (to the best of our knowledge), we include it here as Lemma 3. 

The paper is organized as follows: in section [n] we recall the salient features of dynamic cavity method applied 
to dilute Ising spin systems; in section |III| we consider fully asymmetric systems and state and prove Lemma 1 and 
Lemma 2; and in section [TV| we consider symmetric systems and state and prove Lemma 3. In section |V| we briefly 
summarize our results. References to the earlier literature, especially as pertaining to other methods to analyze the 
systems under consideration, are given where appropriate throughout the paper. 



II. MICROSCOPIC DYNAMICS FOR ASYMMETRIC DILUTE ISING MODELS 



The asymmetric dilute Ising model is defined over a set of N binary variables a = {a±, . . . , <tat}, and an asymmetric 
graph G = (V, E) where V is a set of N vertices, and E is a set of directed edges. We use the notation fully asymmetric 
when if there is an edge (vi,Vj) there is no edge (vj,Vi), and symmetric when if there is an edge (v%,Vj) there is also 
an edge (vj,Vi). A symmetric diluted Ising model is hence here a special case of an asymmetric dilute Ising model. 
To each vertex vt is associated a binary variable er^. The graphs G are taken from random graph ensembles with 
bounded average connectivity c. 

The microscopic description of the dynamics of such system with an synchronous update rule is a Markovian 
dynamics for the evolution of the joint probability distribution 



t 



p(a(Q),...,a(t)) = l[W(a(s)\h(s))p(a(0)) (1) 



where the transition matrix W depends on local fields associated to spins denoted by h 



h i (s) = J2j ji a j (s-l) + 6 i (t). (2i 

jedi 

and the local fields determine jump rates 



Wi((n(t)\hi(t)) = 1 (3) 

1 + exp{2pai(t)hi(t)) 

In a synchronous update rule, one, some or all the spins are updated in each time step. We will here consider the 
two extreme cases, where either all spins are updated (parallel update rule), or where just one randomly chosen spin 
is updated (sequential update rule): 

W(Pt(f\\h(oW-l n£=i Wi(di(t) | hi(t)) parallel update 

{ { ) 1 [ >> ~ 1 1/iVEi n, ¥i <Wwt-x) M^(t) I h(t)) sequential update W 

We note that the sequential update rule is not the same as asynchronous updates (Glauber dynamics), because the 
decisions of which spin is chosen and whether/ whether not/ to flip that spin are here taken in the opposite order. 

Equation ([I]) can be marginalized over one spin i, or over the neighborhood of that spin di. The probability of 
observing a history of a single spin then follows a self-consistency equation, which for the parallel update reads 

Pt (a l (0), ...,<Ti(t) | 6,(0), ...,0i(i)) = pM(Q)) Pdi(<rj£di{0), . . -,tr jeai (t) | ^(0), a,(t)) 

a j€ai (0),...,ff j€Si (0) 



l[ Wi (°i(s)Ms)) (5) 
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and for the sequential updates 

ft(o-i(0), o-i(i) | 0i(O), = ft(o-i(0)) J! P«(o-jeei(0), • ■ • , ffiewW I ^(0), tr t {t)) 

S j€ai (0),....a j€ai (0) 

8=1 ^ ' 

The variables appearing in the conditional probability pot above are the histories of the cavity spins, and the two 
equations can be considered output equations for the dynamic cavity method. In the Belief propagation approximation 
the histories of the cavity spins are independent conditional on the history of spin i, which means 

Pdi(<rjedi(0), Vj£di(t) | <7i(0), <7i(t)) = Y[ Mj->i(°j( )) • • • > ki(0) • • • <Ti(t)) (7) 

jedi 

where fi denotes a marginal probability of the history of cavity spin j, conditioned on the history of spin i (dynamic 
BP messages). Note that in Eq. [6] and Eq. [5] the set of spins contributing in the trajectory of spin i are those with a 
directed incoming link to spin i. 

The evolution of marginal probability at time t can then be obtained by summation over the past history, i.e, 
Pi(<Ji(t)) = Ylo-ifo) adt) Pi( a i(Q)i ■ ■ • : a i(t))- It is straightforward to verify that in general the evolution of Pi(oj(i)) 
requires information from the whole past history and therefore is a non-Markovian process. A main result of [7] and [T] 
was that by a a further assumption of time factorization of the dynamic BP messages, the evolution of Pi(<Ti(t)) is 
a Markov chain of order 2 (i.e. the evolution requires information on one and two time steps earlier). Intuitively, it 
may be argued that for fully asymmetric models, where if dynamic BP messages go out they don't come back unless 
going around a long loop in the graph, the Pi(ai(t)) should obey Markovian dynamics, without the assumption of 
time factorization. In the following section we will show that this indeed is the case for parallel updates, where time 
factorization always holds - but it is not true for sequential updates. 

We end this section by a remark on random graph ensembles. In l] we followed the parameterization of [2] using 
a connectivity matrix Cy, where c.y = 1 if there is a link from vertex i to vertex j, c^j = otherwise, and matrix 
elements and Cki are independent unless {kl} — {ji}. In this parameterization the random graph is specified by 
marginal (one-link) distributions 

P(.Cii) = jj8 1 , Cii + (l-j-)6o,c ij ■ (8) 

and the conditional distributions 



p(cij | cji) = eS CijtCii + (1 - e)p(cy) 



(9) 



In this model the average degree distribution is given by c, and the asymmetry is controlled by e g [0, 1 ]. The results 
given below describe e = (Lemma 1 and Lemma 2, section III) and e = 1 (Lemma 3, section III). In the first case 
the analogy is however only exact in the limit of large system size. 



III. FULLY ASYMMETRIC NETWORKS 



In this section we assume fully asymmetric diluted Ising models such that if spin i is connected to spin j then spin j 
does not connect back to spin i. This property simplifies the evolutionary equations of single site probability because 
influences (through interactions) do not return. 

We consider the two update rules separately. 

A. Fully asymmetric models — parallel update 
Lemma 1 The following recursive equation holds for the fully asymmetric networks 

p(a i (t))= E ^^(t-D) 2cosh( ^ w) (10) 
(?j 68i (t-i) 
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The lemma hence states that the evolution of single site distribution in parallel update follows a Markovian process 
when the network is fully asymmetric. Therefore at each iteration we only need to have information about one 
iteration step before. 



Proof The proof is a straight-forward consequence of the definitions. For this update rule the marginal probability of 
spin i at time 1 (after the first update) is pi(<Ji(l)) = ^ CT . . pQi(<jQi(0))wi(ai(l)\hi(l), where the marginal Pdi(cdi(0)) 
is given by the initial conditions. For the marginal probability after t steps we have 



E E p(**(o),-,w*-i))ri 2^^) < n > 

<T 4 (0),...,<r 4 (t-l)<7ei(0),...,<78i(t-l) s=1 V V " 

Since the neighborhood di denotes the set of spins connected to spin i by a link incoming to i, and since in fully asym- 
metric models such spins will not be connected to i by a link outgoing from i, the probability pgi (agi (0) , aj e gi(t—l)) 
is in this case independent of the history of spin i. We can therefore sum over time, and the only remaining term is 
the joint probability distribution of the cavity spins one update before t. 

e /3<r 4 (t)M*) 

= E - D) 2 cosh(^(t)) (12) 

a-jeai(t-l) 

End of proof 

It is worth pointing out that the above conclusion is true in general and does not rely on the Belief propaga- 
tion approximation. Indeed, we have not used the BP approximation in the calculations above. The corresponding 
output equation for dynamic BP reads 

pm*»= e n^(^(t-D) 2 as) 

^eei(t-l) jedi yH u " 

The dynamic BP messages themselves obey the following recursion equations 



P<Ti(t)h\ S \t) 



iH-+j(*M)= E II Hk^Mtt-i)) n _ {j) (14) 



WW)^ 2cosh(/^)) 

Ai 



where ft^ is the effective field on spin i in the cavity graph, h{ = J2kedi\j Jki&k(t — 1) + 



B. Fully asymmetric models — sequential update 

Lemma 2 For sequential updates, the time evolution of marginal probability distribution does not follow a 
Markovian process, and generally depends on the whole history. 

Proof The proof proceeds by showing that a reduction analogous to the proof of Lemma 1 above does not 
take place for sequential updates. The marginal probability distribution after one step is given by 

Pite(l)) = E E Pfoefli(O)) (1 W ,M1)|M1)) + (1 - ^)<Wo W i)) PiMO)) (15) 

Performing the summation over <7j(0) = { — 1, 1} will split this equation into two parts 

ViW)) = i £ K^(O)HMl)IMl)) + (1 - i)pj 0) Ml)) (16) 

<r a4 (0) 

The last term is the probability distribution p i f\ai{\)) over spin i at time 0, but taking as argument the value of 
spin i at time 1. It is clear that this term is problematic, and we will show that this problem does not go away. After 
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two iterations we have 

PifoPO) = Jp E P(^(l))^(^(2)|^(2)) 

+ E p(^8i(l)K(o-i(2)|fti(2)) 

+ ^^-^ E P(^(l)M(^(2)|Ai(l)) 

+ (1-^)V(^(2)) (17) 

The first two terms partially cancel, but that is all. Therefore, the equation for the evolution of marginal probabilities 
at iteration step t contains all sequences of possible update series 



iV 

+ (i-i)V ( CTi (t)) 

+ ^7-(0,...,t-l) (18) 

where the first term corresponds to the case where spin i has been updated all the time, the second term is for the 
case where it has never been updated and the last term stands for all permutation of different update trajectory in 
which none of the first two cases happen. 
End of proof. 



IV. SYMMETRIC NETWORKS 

We begin by introducing the time factorization ansatz for models which are not fully asymmetric. In both cases, 
these amount to the assumption 



Mi-»ifo(0), . . . , aj(t) | ^(0) . . . aft - 1)) = /4°!i(^(0)) II »j S \i(°j( s ) I °i( s - !)) 



(19) 



and lead to respectively 
for parallel updates, and 



E II - !)) vM*) I hPWp^Wt 2)) 

3 ai (t-l) kedi 



PifoC*)) = ^P< _ Vi(*)) + (1-^) 



e n /*u>fc(*-i)i^ } (*-2)) 

(Ji(t-2) ,3 0i \j(t~l) fcedi 

W i {a i [t)\h i {t))p t r 2 (< H (t-2)) 



(20) 



(21) 



for sequential updates. The numerical results reported in [T] are based on these equations. For fully asymmetric 



models, Eq. 20 reduces to Eq. 13 



In the following we will discuss the fixed points of (20) and (21) - which are obviously the same - for symmetric 



networks, and show that the fixed points of ordinary Belief propagation also solve these equations. This property was 
stated in [7] and, from the viewpoint of generating functional analysis, already seven years ago in A proof has 
however to our knowledge not appeared based on dynamic cavity formalism.. 
Lemma 3 In stationary state, the ordinary BP equations satisfy Eq. [20| and Eq. |2l| 
Proof: Introducing the usual cavity fields for the dynamic messages, /Ltf_>. Jai(t)) 



2 cosh(tii^j (£)) 



we can rewrite 
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Eq. [20] in terms of cavity fields. They fulfill the following equations 



, i r ,, v ^ P [^»'(E tea , v j ti »r'+"i 



7 a .. . ,(T ■ 



;k£dj\i ' 



n 



exp[/3((T* 1 u t k _ 



t-2„.t— 2 1 



Hem 



aT*J kj )] 2cosh[/3 ( j 



t-2 t-2i 



(22) 



where the variables are indexed by spin number and time. These equations can be simplifies using the following two 
well-known formula 



where 



2cosh[/3(u + Jo-)] = c(u, J) exp((3V(u, J) a) 
cosh(/3u) cosh(/3 J) 



c(u, J) = 2- 



cosh(/3V/(M, J)) 



V(u, J) = — atanh[tanh(/3J) tanh(/3u)] 

P 



The factorized normalization term in the Eq. p2|) then reduces to 



n 



feeaA .2cosh[/?«^ 



■o\ 2 J k j 



n 



.kedj\i C ( J jk> u k^ 



ex P (^*- 2 ^ m-fc»«tio) 

kedj\i 



(23) 

(24) 
(25) 

(26) 



Equation (22) using ([26j) is not ordinary BP equations, but we can show that it admits fixed points of ordinary BP 
as a fixed point. We first assume that Eq (22) is at a fixed point, so that the time indices can be ignored. Then we 
interpret the messages in Eq. (22) as ordinary BP messages, and compare to the BP fixed point equations for the 
diluted Ising spin glass: 



E v ( J ]k,u t k J!; j ) = ^2 ■^atanh[tanh( i SJfcj)tanh(u fc _yj)] = 9 3 +■ 



k£dj\i 



k£dj\i 



(27) 



It is seen that the solutions to Eq (27) are then also solutions to Eq (22). 
End of proof. 

The ordinary BP equations are not necessarily the only solution to the dynamic cavity equations in the time- 
factorization approximation. It would be of interest to investigate whether the temperature in which ordinary BP 
starts to fail coincides with the temperature where dynamic cavity equations do not converge to a fixed point. We 
plan to return to this point in a future contribution. 



V. CONCLUSION 



The dynamic cavity method is a way to compute (approximately) marginals of non-equilibrium states. It has recently 
been shown by us and others to be exact in certain cases, and surprisingly accurate in a larger class of models. Since 
computing marginals of non-reversible Markov chains is a rather general problem, it is clearly important to outline 
when these methods can be expected to be accurate, and/or exact. In this paper we have looked at these questions for 
fully asymmetric models, for parallel and for sequential updates, and for symmetric models. A major open problem 
at the moment is if this approach can be extended from synchronous to asynchronous update rules. 

We end by a short discussion where these methods could be useful. First, non-equilibrium physical systems live 
in finite-dimensional space, and have (on the lattice) factor graphs with many short loops. This is therefore not 
a setting where the dynamic cavity method would be expected to be competitive. Applications should instead be 
sought in systems (social, technological, biological,. . .) which can reasonably be modelled by sparse random graphs 
or hyper- graphs. One such application could be describing bargaining processes to reach agreement through local 
interactions, as in the majority game for consensus investigated in [?]. Another could be describing networks of queues, 
which, in contrast to standard queueing theory, do not obey a partial balance condition Models of this kind were 
investigated numerically some time ago to determine blocking probability in certain types of mobile communication 
systems [10], and dynamic cavity method could be of relevance to speed up such estimations. A third could finally be 
to improve upon network inference algorithms of the "kinetic Ising" type [31 [HI [SI H2] through more accurate estimates 
of the direct problem. 
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